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Abstract 

The No Child Left Behind (NCLB) legislation in the Unite States is not equivalent to a national curriculum; 
however, it does set forth some conditions essential for enhancing student achievement at scale (e.g., curriculum 
standards, assessment standards, and expectations of compliance). The missing component within these conditions is 
support for teachers in translating curriculum standards into instruction so students are fairly assessed through the 
NCLB state assessments. This paper examines some commonalities with the Education Reform Act of 1988 and the 
National Literacy Strategy (NLS) implemented in England in 1998 and reports on the development, implementation, 
and effectiveness of the Blending Assessment with Instruction Program (BAIP). BAIP is a program designed to 
provide support to teachers in aligning their instruction with indicators that operationally define curriculum based 
standards in mathematics. The online lessons for teachers are supplemented by online instructional tutorials for 
students that are aligned with each lesson, designed to develop competencies in the skills, and knowledge required 
by indicators and standards in grades 3, 4, 5, 6, 7, 8, and high school. BAIP includes a feedback system that 
immediately provides individual student and aggregate performance data to teachers to enhance instructional 
decisions. BAIP has been beta tested in 187 schools districts with 88,700 students. 

Introduction 

The international emphasis on improving educational outcomes for elementary and secondary 
learners in public schools has taken different forms. However, there are common elements. Some 
have been driven by initiatives that are political in nature; others are culturally and/or 
philosophically driven. There is also the view that the impact of the comparative performance of 
students among countries has stimulated the movement. As a result of these influences, the 
pedagogical principles consistent with the views and preferences of the professional community 
are not always evident, and in some cases give rise to public criticism The range of models that 
have emerged over the past five years tend to approximate some aspect of two versions (e.g., 
prescribed national and state curricula and a standards-based model). Two examples that are 
embedded in legislation include the national curriculum in the England as defined by the 
Education Reform Act of 1988 and the National Literacy Strategy (NLS) implemented in 1998 
and the No Child Left Behind (NCLB) legislation passed by the United States Congress in 2004. 
Central to this movement has been a major effort by most countries to create resources for 
teachers that are aligned with the curriculum approach they have adopted for enhancing the 
academic achievement of their students. This paper focuses on a major intervention in 
mathematics that has been developed at the University of Kansas as one approach to assisting 
teachers in the United States in providing instruction for their students in meeting the standards- 
based expectations of NCLB. As a context for describing the Blending Assessment with 
Instruction Program (BAIP), the common elements of the national curriculum model employed 
in England and the standards-based model in the United States will be discussed. The standards- 
based model embedded in NCLB is a framework for discussing some of the lessons learned in 
the United States. This will be followed by a presentation of the work that has been on-going for 
the past four years at the University of Kansas on blending assessment with instruction, a large- 
scale intervention designed to enhance attainment of the NCLB requirements at both classroom 
and state levels. 
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When one examines the under girding legislation and the history of the two different approaches 
(i.e., England vs. the United States), the commonalities become apparent. Specifically, 


the 


Both are prescriptive, but vary in degree. 

Both are concerned about equity for disadvantaged students. 

Both are accountability oriented. 

Both have consequences for nonperforming districts. 

Both identify core subjects. 

Both require assessments at specified ages or grades. 

Both aspire toward higher achievement outcomes. 

Both include professional development. 

Both resulted from legislation. 

Both are intended to allow teachers to organize their instruction to meet 
needs of their students. 

Both have resulted is mixed views from the public and professional 
communities. 


Curriculum standards, as found in the United States movement, by nature lack the prescriptive 
influence of a national curriculum. However, it is widely agreed that the standards being 
employed in many individual states within the United States represent a step toward a national 
curriculum, particularly when aligned with high-stakes assessments. High stakes take the form of 
legislated policies (e.g., it is widely agreed in the United States that state assessments, as 
translations of the standards, tend to drive instruction that move the curriculum toward a 
common focus). Thus, the consequences of failing to meet accountability policy, as specified in 
the law, puts pressure on teachers to teach to the standards covered by annual summative tests or, 
at least, to place a great deal of emphasis on formative tests made available to teachers for 
instructional purposes. This is not a pressure that is solely felt by teachers in the United States. 
According to Rood (2002), teachers in England report the same type of accountability pressures 
as they prepare students for national assessments. These pressures, combined with the view that 
undue emphasis is placed on the subjects assessed at the expense of other knowledge and skills 
considered important for the futures of learners, have elevated the level of public concern about 
the ultimate impact of these movements on learners and in some case on the instructional 
program in general. 

Achievement by United States Students in Mathematics 

The influence of national comparisons is particularly evident in the United States. According 
to the National Mathematics Advisory Panel (2008), “American students achieve in mathematics 
at a mediocre level by comparison to peers world wide” (xii). This claim is supported by the 
results of the 2003 Trends in International Mathematics & Science Study conducted by the 
National Center for Educational Testing, which found 11 out of 25 countries outperformed the 
United States in mathematics achievement at the fourth-grade level, and 14 out of 45 countries 
outperformed the United States at the eighth-grade level (Institute of Education Sciences 2006). 
Moreover, comparisons between the United States Trends in International Mathematics & 
Science Study test results in 1995 and 2003 indicate that the average math performance of 
students at the fourth-grade level did not improve during the 15-year span between the two 
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assessments. In the United States there is evidence of improvement in achievement, as indicated 
on the National Assessment of Education Progress (NAEP), which has recently reported positive 
trends of scores in fourth and eighth grades (National Mathematics Advisory Panel 2008). 

In the United States the emphasis is placed on applying the standards and assessments to all 
students including those with disabilities. This adds to the challenge of districts in demonstrating 
Annual Yearly Progress. The NAEP (National Center for Educational Statistics 2005) reported 
that “in 2003 the average scale score difference for students with disabilities and students 
without disabilities was 23. This gap decreased only by 2 points in 2005 when average scale 
score differences for student with disabilities and students without disabilities was 21” . 
Additionally, the National Center for Educational Outcomes reported in 2004 that not only were 
students with disabilities performing below all students across the country, but the achievement 
gap grew significantly larger as students got older (Thurlow & Wiley 2004). 

Furthermore, research now suggests that students with learning disabilities perform poorly on 
standards-based assessments (Thurlow, Albus, Spicuzza, & Thompson 1998). More specifically, 
83% of non-disabled students passed a state assessment on basic math skills, whereas only 34% 
of students with disabilities passed the test. Studies have shown that students with learning 
disabilities typically function two to four grades below expectancy across the mathematics 
curriculum (Parmar & Cawley, 1997). Additionally, these students only attain fifth to sixth grade 
performance levels by the time they graduate from high school. Thus, their study of 1,200 
students with learning disabilities found that these students experienced an age-in-grade 
discrepancy over a six-year period, demonstrating that the discrepancy progressively increases as 
students become older and move to higher grades. 

In addition to the achievement gaps between students with and without disabilities, 
achievement gaps in the United States are also observed between students at risk due to 
economic factors and their peers. This issue does not appear to be limited to the United States. 
According to Shiel (2003), the National Literacy Strategy was put into place, in part, due to 
concerns regarding the quality of instruction, particularly in schools serving economical diverse 
students. Several researchers in the United States have found that socioeconomic status (SES) is 
significantly related to a student’s mathematics achievement (Lauzon 2001; Ma 2005; Okpala 
2002). For example, Lauzon (2001) noted that a student’s SES led to lower-than-average test 
scores. This is supported by Ma (2005), who indicated that white and Asian students of low 
family SES demonstrated a lower rate of mathematical growth than students of a higher status. 
Moreover, when examining the mathematics achievement of fourth grade students during 1994, 
1995, and 1996, Okapala (2002) found that a student’s economic status negatively impacted 
achievement on the North Carolina state mandated end-of-year tests. Similarly, when analyzing 
the Third International Mathematics and Science Study, Ercikam, McCreith, and Lapointe (2005) 
found that one of the strongest predictors of mathematics achievement and participation in 
advanced mathematics courses was a student’s SES. 

Basic Principles of No Child Left Behind 

The United States Department of Education web site (2008) identifies the following as “four 
pillars” of NCLB. 

1. Stronger accountability for results: The focus is on enhancing achievement for 
all students including students who are disadvantaged. State and district report 
cards inform parents. Corrective actions must e taken if progress is insufficient. 
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2. More freedom for states and communities: States and districts are given 
unprecedented flexibility in how they use federal funds. Districts are allowed to 
use funds for their particular needs. 

3. Proven education methods: Programs and practices proven via scientific 
research to improve student learning and achievement receive federal support. 

4. More choices for parents: When a school fails to meet state standards parents 
may transfer their children to higher performing schools. Supplemental services 
must also be provided. 

Under NCLB, states are allowed to establish their own curriculum standards, structure 
assessments that measure student performance, and set attainment levels of successful 
performance. The standards employed by states in mathematics tend to reflect the national 
curriculum standards developed by the National Council of Teachers of Mathematics (NCTM). 
These standards were described as being appropriate when first generated in 1989. However, 
many teachers find them to be abstract and difficult to translate into instructional practices. This 
complicates the process of aligning instruction with curriculum standards with which large-scale 
assessments in NCLB are aligned. 

As a result, the National Mathematics Advisory Council Final Report (NMAC 2008) proposed a 
model based on what students need to know at particular grade levels. NMAC refers to the 
grade-specific outcomes as benchmarks for critical foundations in mathematics. Support for use 
of the NMAC recommendations instead of curriculum standards in mathematics framed in the 
Final Report of the National Mathematics Advisory Panel (2008) has been quick to emerge. For 
example, 50 professors in mathematics at three universities in the state of Missouri have 
recommended that the curriculum standards for mathematics in Missouri be reframed to reflect 
the NMAC recommendations (Catchings 2008). 

Blending Assessment with Instruction Program Construct 

It is in the context of emerging national standards, increasing demands for accountability and 
assessment, as well as public concern about the need for improved instruction, that the Blending 
Assessment with Instruction Program (BAIP) was conceptualized. It represents an initiative to 
enhance alignment of instruction in mathematics with curriculum standards and, in turn, state 
assessments under the prevailing conditions in United States schools as a consequence of NCLB. 
In researching how best to design and produce resources to meet the needs of teachers 
responsible for mathematics instruction, researchers at the University of Kansas developed two 
sets of principles: (a) principles related to prevailing conditions in mathematics instruction and 
(b) principles related to access and adoption. These principles provided the construct for 
instructional design, technology interface decisions, and the architecture for the management 
system. 

Principles: Conditions in mathematics instruction 

1. Instructional Alignment: An underlying assumption of BAIP is that students’ failure to 
perform at prescribed performance levels is not necessarily a learner attribute, but represents a 
failure to translate curriculum standards into appropriate instructional resources to assist teachers 
in aligning their instruction with the curriculum standards that govern the development of state 
assessments. For instructional alignment to occur, teachers must understand the curriculum 
standards and be able to translate them into instructional practice. Given that curriculum 
standards are the cornerstone of NCLB, it is reasonable to expect that the standards are known 
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and understood by teachers. However, many teachers lack an understanding of curriculum 
standards in mathematics. Thus, Maccini and Gagnon (2002) found that 5% of general education 
teachers and 45% of special education teachers who teach math were not familiar with the 
NCTM standards. In addition, their study found that only 59% of special education teachers and 
78% of general education teachers felt confident about teaching the mathematics standards that 
the NCLB tests are designed to assess. 

The consequence of failing to achieve instructional alignment is that many students are 
subjected to assessment over content, concepts, and skills that they have not been taught or 
exposed to. Not only does this put some students at a disadvantage, it also points to a weakness 
in the model many states apply when implementing NCLB. The availability of curricula aligned 
with standards has a positive effect on classroom practices (McGehee & Griffith 2001). Without 
curriculum alignment, students, especially those who are at academic risk or have a disability, 
face an unfair disadvantage on state assessment tied to NCLB accountability (Wiley, Thurow, & 
Klein 2005). 

2. Teacher Content Knowledge: Even when teachers are familiar with the curriculum 
standards on which assessments are based, they still have to make decisions about how to 
translate the standards into strategies for teaching. This is particularly critical at the elementary 
level, where teachers tend to lack the needed content knowledge. In contrast to teachers at the 
middle and secondary levels, where licensure requirements dictate preparation as math 
educators, elementary teachers are prepared in several areas of pedagogy in addition to teaching 
mathematics (e.g., science, reading, and social studies). As a result, many are less prepared in 
content knowledge in mathematics. Additionally, a significant portion of elementary level 
teachers lack confidence in teaching mathematics. This adds to the need for instructional 
resources that are aligned with curriculum standards and offer detailed guidelines to assist 
teachers in making instructional decisions on how best to translate curriculum standards into 
effective teaching strategies. 

3. Instructional Planning: Effective instructional planning is central to meeting the 
requirements of NCLB. Driven by standards, the law places teachers in the position of having to 
plan at a level that traditionally has not been characteristic of most teachers in the United States. 
That is, they must bring to the planning process not only the skills essential to instructional 
planning, but also the content knowledge necessary to make the decisions inherent in translating 
standards into instructional strategies. Research on differences in how teachers are prepared in 
China and the United States report that while pre-service teachers in China have less preparation 
in mathematics than their counterparts in the United States, their mathematics knowledge is more 
profound, more flexible, and more adaptive (Ma 1999). The differences between instruction and 
mathematics content knowledge has been attributed to the amount of time teachers in China 
devote to developing, reflecting on, and modifying daily instruction. Specifically, in China 
teachers spend about four hours a day in planning (Shen Sue & Fan 2007). 

4. Explicit Instruction/Guided Practice: According to the Final Report of the 
National Mathematics Advisory Panel (2008), students who have mathematical difficulties 
benefit from explicit instruction that (a) models problem-solving skills using a variety of 
examples, (b) allows time and support for guided and independent practice, and (c) provides 
students with extensive feedback. This is backed by years of research showing that effective 
teaching occurs when teachers present material in small steps, model procedures, and check for 
student understanding (Rosenshine 1997; Rosenshine & Stevens 1986). Similarly, Mastropieri, 
Scruggs, and Shiah (1991) found, after reviewing 30 research studies in the area of mathematics, 
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that variation of modeling, demonstration, and feedback improved and maintained the 
acquisition of math skills. 

5. Findings: The findings of the National Math Advisory Panel regarding the 
impact of practice and feedback are not new. The effectiveness of feedback in learning can be 
traced back to studies conducted by Thorndike. More recently, Mory (2004) has summarized the 
history of research on the effectiveness of feedback. Further, when evaluating the effects of 
immediate or delayed feedback in the acquisition of addition, subtraction, multiplication, and 
division facts, Brosvic, Epstein, and Dihoff (2006) found that immediate feedback over delayed 
feedback aided third-grade students in acquiring math facts. Moreover, the role of guided and 
independent practice in eradicating learning errors and misconceptions before they cause lasting 
effects has been well documented (Hunter 1994; Joyce, Weil, & Calhoun 2004). According to 
Hunter (1994), guided practice can help ensure that students can complete tasks independently 
without practicing continual errors. A study by Hudson (1997 ) found that guided practice 
following short lecture segments helped students with learning disabilities retain content 
information better than independent practice alone. 

Clearly, if we expect to be able to reach all students and teach all of the objectives and 
standards, instruction must be carefully planned (Watkins & Slocum 2004). That is, time must be 
dedicated to plan how to incorporate directed instruction, guided practice, and feedback into 
daily instruction. One way to accomplish this is to develop precisely planned instructions or 
lessons laid out in lesson scripts. Lesson scripts are written narratives outlining what teachers 
should say to teach or convey academic information, what questions teachers should ask their 
students, and what answers students should provide (Gunter & Reed 1997). By providing 
instruction via lesson scripts, teachers can be certain that all students have access to the same 
content through explicit instruction that provides opportunities for guided practice and feedback. 

Principles: Access and Adoption 

1. Supplemental: Most districts have a curriculum that teachers are responsible for 
implementing. Typically, therefore, while no new curriculum is needed, supplemental resources 
that aid teachers in the instructional alignment process may be needed. Such resources must be 
coordinated with the respective state standards and in a format that is easy for teachers to use. 
Further, since teachers vary in their need for additional resources in the instructional alignment 
process, supplemental resources should be in a format that helps teachers decide which elements 
of the resources are most valuable to them. 

2. Professional Participation in Development : Teachers were viewed as a significant 
resource in the process of developing BAIP resources. The challenge is to find the best teachers 
with the appropriate development skills. The primary strategy employed in selecting writers for 
participation in the development of BAIP was to look for evidence of student achievement at the 
building level as measured by state assessments over a period of time (e.g., three years). This 
was viewed as evidence of teaching effectiveness. Teachers were then asked to serve on writing 
teams. A team was comprised of a teacher at grade level and one grade below. 

3. Integration with State Assessment Strategies: In many districts today, teachers experience 
pressure or at least high expectations that their students will perform well on state assessments. 
The publication of AYP results at the community and state level in the media adds to this 
pressure. Consequently, it is only reasonable to expect that teachers will be responsive to 
resources and opportunities that enhance their efforts to meet the instructional needs of their 
students. Teaching to the test is not the answer. Instead, structuring supplemental resources in a 
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manner that reflects their alignment with the curriculum standards will allow teachers to make a 
judgment about the value of the resources relative to their needs. 

4. Self-Contained Resources: Teachers have little time to search for resources on top of all 
their other daily demands. Therefore, in developing supplemental resources, it is important to 
make them as self-contained as possible rather than directing teachers to a myriad of related 
resources. 

5. Teacher Feedback: If supplemental resources involve student assessments, it is important 
that the results be made immediately available to teachers. Current technologies allow for results 
to be instantaneously transferred to teachers. In an environment where teachers are asked to 
make decisions based on student performance, they are greatly disadvantaged by delays in 
providing student performance feedback on instructionally related measures. 

6. Scalability: The resources of time, expertise, technology, and the expenses associated 
with developing resources to aid teachers nationally combine to make scalability an essential 
development goal. The challenge inherent in achieving scalability involves allowing teachers the 
flexibility to adapt resources to their particular needs and to complement the curriculum they are 
implementing in their classrooms. 

7. Usability: Inherent in the need to make resources available at scale is the importance of 
ensuring, via an intuitive format, (a) ease of navigation, (b) access to resources in a manner that 
allows teachers to customize lessons, (c) the capacity to print any aspect of the resource(s) in 
accordance with teacher needs, (d) the ability to preview material such as PowerPoint slides or 
handouts to determine if they need to be customized to the learning characteristics of a given 
group of students, and (e) the ability to download resources onto the user’s computer desk top. 
The BAIP model that evolved from these principles consists of 276 lessons for grades 3, 4, 5, 6, 
7, 8, and high school. They are aligned with the curriculum standards in a mid-western state. The 
lesson plan design is research based and is intended as a resource for teachers as they develop 
their own lesson plans. Each lesson is accompanied by two online tutorials aligned with the 
lessons and made available to students at any time. Further, a management system enables 
teachers to monitor individual and group progress as reflected in the performance of their 
students, thereby enhancing their ability to make appropriate instructional decisions. 

BAIP Resource Design 

After two years of development, the lessons and tutorials were pilot tested in the spring of 
2007 with 2,000 students in grades 5, 6, 7, and 8. After making revisions based on the results of 
the pilot test, the materials were subjected to large-scale beta testing during the spring of 2008. 
The beta test involved 187 school districts and 83,551 students in grades 3, 4, 5, 6, 7, 8, and high 
school, including 3,777 students with disabilities. The following lesson and tutorial descriptions 
reflect the designs subjected to the beta test. 

Classroom Teacher Lesson Design 

The lessons are tied to indicators that operationally define each standard, with approximately 15 
indicators per standard. Lessons are not intended to represent a curriculum but to serve as 
resources to help teachers align their instruction with the skills and concepts on which prevailing 
assessment standards are based. The lessons are also designed to help teachers respond to 
specific learning needs that have been identified through formative assessment data from online 
standards-based tutorials. 
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The lessons have been designed to be accessed online. As such, they can be reconfigured into 
personalized lesson plans or used as presented, and any aspect of a lesson can be printed if a 
teacher prefers to work from a hard copy. Templates were developed for each feature of the 
instructional design for ease of production for web-based delivery. Writing teams comprised of 
master teachers developed the lessons at each of the targeted grade levels, and each lesson was 
subsequently reviewed by Subject Matter Experts (SME) in mathematics as part of a jury 
process. 

The BAIP lesson design is based on five frameworks that are essential for effective learning: 
Contextual, Teaching, Lesson, Application, and Extensions. 

The Contextual Framework serves as an introduction to the lesson by providing basic 
information about the standard, benchmark, and indicator pertaining to the lesson (see Fig. 1). 
The Teaching Framework provides further details about the indicator and serves as an 
instructional tool designed to increase the teacher’s knowledge about the specific indicator. 

Fig. 1. Contextual framework of a seventh-grade lesson. 
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The Lesson Framework begins with key prior knowledge concepts that students must have 
mastered and strategies for reviewing these skills in connection with the new knowledge. Thus, it 
serves to link students’ understanding, experiences and background knowledge to important new 
concepts (Falk 2000). According to the NCTM (2000), “in a coherent curriculum mathematical 
ideas are linked to and build on one another so that students’ understanding and knowledge 
deepen and their ability to apply mathematics expands” ( 2). Therefore, the lesson identifies and 
provides instructional strategies for key prior knowledge and skills related to the concept. The 
Lesson Framework also provides teachers with an age-appropriate application of the new 
concept and/or skill, which is crucial to achieving deep understanding. This age-friendly 
description of the concept and/or skill can also serve to motivate and capture students’ interest 
(Robinson, Robinson, & Maceli 2000). 

Finally, the Lesson Framework provides teachers with specific strategies to introduce concrete 
models of a concept and/or skill, followed by step-by-step demonstrations of the mathematical 
processes. Each of the strategies is described explicitly in terms of teacher prompts and 
corresponding student responses. The prompts serve two purposes. First, cueing and choral 
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responses require students to be actively engaged in the lesson (Little 2003). Second, students’ 
responses allow the teacher to continuously monitor students’ knowledge and skill acquisition. 
The online tool allows teacher access to training on the lesson model and describes how each 
feature works, in addition to presenting the lessons to the teacher within an e-leaming design. 
The lessons are self-contained such that teachers are not referred to other web sites to search and 
review resources as part of using and /or modifying the lessons. This model was designed to 
minimize the time required on the part of teachers to integrate the lesson, concepts, related 
strategies, and curricular standards as part of their instruction. 

The Application Framework follows a three-step process of (a) guided practice, (b) 
independent practice, and (c) open-ended validation questions. BAIP provides teachers with all 
the materials necessary to perform these three steps, including practice sheets and a grading 
rubric to be used with the open-ended validation questions. Additionally, prompts are provided 
for teachers to utilize in guiding students through the initial guided practice activities. 

The Extensions Framework provides teachers with additional activities for students with 
learning disabilities and/or any students struggling with the concept. These activities range from 
critical skills that might need to be reviewed for students to advanced applications of a given 
indicator. 

Fig. 2 illustrates the organization and content of the lesson design. Each feature in the outline 
represents an interactive feature of the online version of the teacher lessons. 

Fig. 2. Outline of lesson design format. 
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Tutorial Design 
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The web-based tutorials are designed for independent study and to provide immediate feedback 
to the student. According to Martindale, Cates, and Qian (2005), appropriate use of technologies 
such as web-based resources contributes to effective learning. In fact, when evaluating a web- 
based instructional tool designed to provide interactive practice aligned with Florida’s Sunshine 
State Standards, Martindale, Pearson, Curda, and Pilcher 2005) found that fifth-grade students 
who used the tool scored significantly higher than those who did not use it in mathematics on the 
Florida Comprehensive Assessment Test . In addition, McDonald and Hannafin (2003) showed 
that third-grade students who interacted with a web-based review program scored higher on the 
State of Virginia’s Standards of Learning Exam than students who did not interact with the 
program. 

The intent of the BAIP student tutorials is to help students understand the skills and concepts 
required to achieve the indicators associated with standards. To that end, they incorporate 
assessment items stated in a format similar to that used in state assessments. Students begin the 
tutorials by selecting an indicator related to a standard. Once the indicator is selected, the 
mathematics content within the indicator is introduced in student-friendly language. Next, the 
student is presented with a mathematical problem and instructed to choose the correct answer. 
(See Fig. 3 for a sample screen.) The four response choices for each item are based on three 
common misconceptions of mathematics principles that students exhibit in learning math 
concepts. If students select an incorrect response, they are given a brief explanation of why the 
choice was incorrect. Further, students are offered a guiding question on the next item to target 
their thinking before they attempt another item measuring the same concept. The tutorial then 
provides one more equivalent practice items and a final “challenge” item, which involves a 
slightly different context or problem, but the same content/skill. The ultimate focus is on 
students’ misconceptions and targeting the instruction to enhance students’ knowledge so that 
they learn why a given response is correct or incorrect. Directing students through an adjusted 
thinking process before a second attempt is critical to correcting misconceptions linked to a 
specific content topic. 

BAIP moves well beyond prescribing the same general instruction to all students based on 
the erroneous assumption that all students missed the same item for the same reason. Thus, the 
tutorials not only provide students with formative feedback, but also provide teachers with 
formative feedback about student misconceptions and gaps in understanding of mathematics 
knowledge and skills. These formative assessment data can be used by teachers to make data- 
driven decisions regarding classroom instruction. 


Fig. 3. An example of a practice problem screen. 
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D. 156 
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If students answer the four problems correctly, they are given the option to move on to 
another indicator. If they miss one of the four problems, the web-based tool notifies the teacher 
that they did not answer the questions correctly and reroutes students back to the key math idea 
so that they can try the problems again. Students have two opportunities to answer the problems 
correctly before being routed to another indicator. 

Fig. 4 illustrates the steps through which students progress as they complete a tutorial. 

Fig. 4. Steps of the web-based tutorial. 

Steps of the Web-Based Tutorial 

Step 1: Selection Screen: Students are presented with a list of concepts covered in a standard and asked to select a concept. 

Step 2: Key Math Idea: A brief description of the indicator associated with the standard to be learned and practiced is provided. 

Step 3: Instructional Problem: Students are asked to answer a question based on the concepts covered in the indicator introduced 
on the previous screen. 

Step 4: Instructional Explanation: An explanation of the mathematical process that must be followed to figure out the correct 
answer for the problem presented on the previous screen is provided. 

Step 5 : Practice Problem-1: A new mathematical problem over the same content is presented and students are asked to answer it. 
Students are told if they correctly answered the question or not. An explanation on how to solve the mathematic problem is 
provided. 

Step 6: Practice Problem-2: A new mathematical problem over the same content is presented and students are asked to answer it. 
Students are told if they answered the question correctly or not. An explanation on how to solve the mathematic problem is 
provided. 

Step 7: Practice Problem-3: A new mathematical problem over the same content is presented and students are asked to answer it. 
This question is designed to help ensure students have learned the concepts and can generalize what they have learned. Students 
are told if they answered the question correctly or not. 

Step 8: End of Tutorial: If students get all four questions correct, they are routed to the selection screen to select a new indicator. 
If they do not get all four questions correct, data are sent to the teacher and students are re-routed to the beginning of the same 
tutorial. 


Throughout, BAIP underscores the importance of data-based decision making on the part of 
teachers. Thus, teachers receive records of how well each student is doing on the tutorials for use 
in instructional planning, including which lessons they might want to emphasize in their 
teaching. Data are aggregated by individual students and classrooms. 
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Lessons Learned During the Development and Beta Testing of of BAIP 

The staff of the e-Leaming Design Lab (eDL) at the University of Kansas, where the design, 
development, and production of BAIP occurred, along with the management of the beta test, is 
experienced in developing large-scale online resources for practicing teachers as well as teacher 
education. The standard development and production processes generalized to the development 
of BAIP resources. Beta testing processes specific to BAIP were designed largely through the 
Center for Educational Testing and Evaluation (CETE) at the University of Kansas with 
logistical support from the Kansas State Department of Education. The focus of the beta test was 
largely directed toward seeking formative and fidelity information applicable to modifying 
and/or enhancing features of BAIP, validating implementation strategies and assessing the 
effectiveness of BAIP in an unobtrusive manner. The latter was important as the goal was to 
engage teachers in using BAIP and keep the obvious nature of research to a minimum. A major 
test of BAIP will occur following the revision process. 

Lessons Learned in BAIP Lesson Development 

Lessons learned during the content development process for lessons that have implications 
for further development and that were responded to during the formative process include the 
following: 

• Assessment, subject matter, technology, instruction, and content management expertise 
are central to creating BAIP-type products. 

• Writing lessons against a research-based model is difficult, but effective. The classroom 
teachers on the writing teams, while knowledgeable about the lesson design features, were not 
experienced in writing lessons in the detail required by the instructional design for BAIP lessons. 
This resulted in the need for additional training and implementation of a systematic review 
process focusing on content accuracy and compliance with the instructional design. 

• Activities are easier to write than teaching concepts. This became particularly apparent in 
the development of lessons for grades 3, 4, 5, and 6, which involved experienced elementary 
teachers. It was not as evident at the middle and secondary grades. The reason for this difference 
seems to be related to team members’ preserves preparation. That is, elementary teachers are 
prepared to teach a wide array of subjects, including mathematics, whereas at the middle and 
secondary levels teachers who teach math are truly trained to be math educators. 

• To ensure standards are understood and translatable into instructional practices, the focus 
must be on teaching concepts. This was reinforced through the feedback provided to writers by 
the subject matter experts during the review process. 

• The language of math used by the writers was not uniform, especially at the elementary 
level. This lack of consistency was solved through an editing process. Specifically, an extensive, 
essential vocabulary was developed for each lesson and extended into the design of the tutorials 
to ensure the use of acceptable mathematics terms. 

• Aligning teaching concepts and applied examples to indicators is important to teachers. 
This became apparent during the content development stage of creating the lessons, where the 
lesson writing teams, comprised of more than 60 carefully selected classroom teachers, provided 
formative input about their perceptions of the instructional design. 

• Teachers appreciate the self-contained nature of the resources. As for the alignment 
feature, the writers were quick to confirm that teachers would value the self-contained model. 
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Having all resources necessary for teaching a lesson provided within the lesson format saves 
teacher time. 

• The use of templates and validated processes for content management and moving 
content to online production is important. We also found early in the data entry of lesson copy 
that it was helpful if data entry personnel were familiar with mathematics. This reduced the 
number of errors resulting from the data entry process. 

The scalability requirement and the magnitude of the development project dictated the 
development of a lesson builder tool to automate many of the technical development processes. 

Lessons Learned in Tutorial Development 

The following lessons learned during the tutorial development process were found to have 
major implications for designing tutorials applicable across ability levels. 

• The process of writing tutorials required skills not needed in writing lessons. This was due 
largely to the assessment items that were central to the tutorial design. 

• Items embedded in the instructional tutorials were stated in the same format as items in the 
summative assessments. This minimized the necessity of learners having to adjust to a new 
assessment format when they took state assessments. 

• Because the tutorials were intended to be completed by students, consistency of format 
design was essential. Consequently, a small number of tutorial writers were used. 

• The focus was on instruction of math skills and concepts. An effort was made to keep the 
reading difficulty of the text below the reading level of the targeted grade level to minimize the 
impact of reading on learning the math skill. 

• Editing was routinely integrated with alpha testing during development so that the content 
was reviewed as part of testing the functionality of the tutorial. 

• Development of the tutorial design and the management system was an iterative process. 
This approach was chosen because value was placed on teachers having immediate access to 
performance data based on the completion of tutorials. 

• It was important for teachers to be able to control the assignment of tutorials and monitor 
student performance to inform their teaching and instructional planning. Consequently, 
significant effort was invested in the management system. 

• The automatic personalization of tutorials for each student added motivational attributes. 

• In order to ensure that students understood the instructional features of the tutorials, graphics 
were used when appropriate. In total, 1,600 graphics were embedded in the 417 tutorials. 

• To accommodate the diversity of learners, teachers insisted on audio capacity for the 
tutorials. This required the creation of 1,600 altags. 

• The cost of creating tutorials is directly related to the extent visualization images and 
animation required. For example, the development of cognitive tutorials is far more time 
consuming and labor intensive than the design ultimately adopted for BAIP. 

Lessons Learned in Beta Test Implementation 

Lessons learned during the implementation of the beta test also have implications in 
preparation for the final release. Examples include the following: 

• Use data can be inferred to indicate the perceived needs of teachers for resources such as 
BAIP lessons and tutorials. Each superintendent in the state received a letter inviting his or her 
district to participate in a 12-week beta test of BAIP along with a description and demonstration 
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site. The same information was sent to principals, curriculum coordinators, and test coordinators. 
The response from districts, without any follow-up, reflected great interest in the BAIP model, 
o 231 superintendents out of a possible 297 expressed interest 

o 187 districts became active users 

o 83,551 students were enrolled in BAIP 

o 3,777 students with individualized education programs (IEPs) were included 

o 80,114 lessons were accessed by teachers 

o 1,206 school buildings participating in 187 districts 

o 574,117 tutorials were completed 

o 94 % of the tutorials attempted were completed 

Following the beta test, a 19-item fidelity survey instrument was sent to each 
participating teacher. A total of 1,051 responses were received without follow-up reminders. 
Examples of the results included the following: 

o Extensions for students with disabilities were considered helpful, but not sufficient, 

o Mini lessons based on state modified assessments were proposed, 

o Training needs to be focused on teachers. 

o Teachers varied in their use of BAIP, in that some used lessons only, some used 

tutorials only, and some employed the system as intended. 

o How BAIP was used depended in part on how they were introduced to BAIP. 

o The trainer of trainers’ model was effective for orientation, 

o Direct teacher training was needed for implementation, 

o A combination of webinars, video, and face-to-face sessions would be helpful. 

The BAIP staff was committed to seeking input from end users, including classroom 
teachers, curriculum consultants, test coordinators, subject matter experts, professional 
development specialists, and others during the development process and beta testing. Prior to 
implementation, seven regional sessions were held in which lessons and tutorials were 
demonstrated and open-ended discussions conducted. An online newsletter was also distributed 
to seek input during the beta testing. Additionally, a help line was established where teachers and 
others could call in with questions or comments. Every effort was made to be responsive as 
quickly as feasible. These avenues were perceived as being important to learning the strengths 
and weaknesses of the BAIP system and suggestions for improvement. A network involving the 
staff and external subject matter experts was created as a means of finding solutions to issues that 
required immediate attention. These efforts, in turn, resulted in relationships that facilitated an 
effective formative approach to development. 

Lessons Learned in Student Performance 

Due to the time of the year, the beta test was only twelve weeks long. As a result, teachers 
varied the amount of lessons they taught and the number of tutorials they assigned. This variation 
and duration limited the conclusions that can be drawn from the results of student performance 
on items embedded in tutorials and performance on state level NCLB tests. However, there were 
trends observed in the available data. Analysis was targeted to grades 4, 5, and 6. 

• The mean number of items in each tutorial answered correctly out of four by students 
with IEPS was 2.32, whereas non-disabled peers averaged 2.53. 

• BAIP was effective across the skill area sub domains and across applications and skills 
representing different complexity levels. 
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• The more tutorials completed, the higher the performance on state assessments of 
students using BAIP than non users. 

• The effectiveness of BAIP in enhancing achievement was positive across the three 
target grades with the highest being at grade six. 

• The feedback from teachers, as reflected in the fidelity survey, indicated a positive 
impact of BAIP on student learning. 

Conclusion 

The Blending of Assessment with Instruction Program (BAIP) is built on the premise that in 
meeting the expectations of a standards-based instructional system, such as that defined by the 
No Child Left Behind legislation in the United States, instruction must be aligned with 
curriculum standards that underlie the summative assessments. To achieve instructional 
alignment with curriculum standards, it is important that teachers not only understand the 
meaning of the standards, but are also able to translate standards into instructional strategies. 
Because schools already have curricula in place, the BAIP resources, designed to provide 
assistance to teachers in the alignment process, were intended to be comprehensive and flexible. 
Thus, they can be used in a supplemental manner. Learner-based resources, such as the online 
tutorials being aligned with the lessons, add an important student experience. Teachers prefer 
that resources be self-contained so they are not directed to other URLs to access the resources 
and also that feedback on student performance is immediate. 

Development is underway for specific lessons for students with disabilities and at-risk 
students as well as resources for parents (i.e., instructional activities, teaching tips and a 
mediated dictionary of essential terms). These resources will be aligned with the lessons and 
tutorials. The management system will also be expanded to accommodate the parent resources 
and place the teacher in control of managing these resources as well as communications with 
parents. A science version of BAIP is currently underway with plans in the future for a BAIP 
version in reading. 
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